This document defines a suggested single rubric for assessing creativity across all MCSI universal rubric categories. It is used by every exercise where creativity is assessed, so that one rubric applies to hundreds of exercises without per-exercise creativity definitions. exercise and category specs reference this rubric and, when needed, supply only evidence_location hints (where to look for each criterion) so instructors know where to find evidence without rewriting level descriptions. A Multi-Eliciting Activity (MEA) is an open-ended assignment that requires the student to use creative methods to solve the problem.
The proposed framework is from the empirical study titled Assessing Cybersecurity Problem-Solving Skills and Creativity of Engineering Students Through Model-Eliciting Activities Using an Analytic Rubric (Kim, et al., 2024) creativity = novelty + usefulness.
The proposed rubric applies to all categories in of the draft universal rubric in development (yara, hunt_query, forensics, report, leadership_deliverable, devops_pipeline, etc.). Documentation and Presentation from the source article are covered by existing critical thinking / execution requirements (methodology, interpretation, required sections); creativity assessment focuses on Originality, Generation & Selection of Ideas, and Value (usefulness).
Levels are described 4 to 0 with distinct differences between each level. The same scale applies to all three criteria. The wording below uses “the submission” and “the procedure” to mean any deliverable (rule, report, code, config, document, analysis) and any described method (methodology, comments, sections, rationale).
| Level | Label | Definition (general) |
|---|---|---|
| 4 | Fully accomplishes | The submission/work fully accomplishes the criterion. It is highly unique (originality), or shows strong generation/selection of ideas and extension beyond the minimum, or fully meets exercise needs with clear justification (value); procedure is clear, justified, and complete; rationales for critical choices are provided and correct; no extraneous information. |
| 3 | Accomplishes with minor gaps | The submission/work accomplishes the criterion. It is unique, involving some new ideas or improvements (they may be predictable or conventional), or shows a tentative attempt to find new uses for common tools or frameworks; meets requirements and is viable; minor embedded errors or rationales that need clarification. |
| 2 | Somewhat accomplishes | Somewhat accomplishes the criterion. Partially meets requirements; has gaps, errors, or lacks clarity. Does not achieve Level 3. |
| 1 | Does not meet minimum | Does not achieve Level 2. The work does not meet minimum requirements for the criterion; critical evidence (e.g. results, rationale, or required output) is missing. |
| 0 | No evidence / No submission | No progress or nothing that resembles a minimal response; no substantive procedure, no submission, or work that does not address the criterion at all. |
The following three criteria are category-agnostic. Instructors apply them to whatever “submission” and “procedure” the exercise produces (YARA rule, hunt query, report, forensics submission, pipeline config, leadership document, etc.). Per-category evidence_location hints (in category_descriptions.md) indicate where to look for evidence without changing these descriptions.
Criterion: Novelty of the submission or approach-unexpected or inventive use of ideas, structure, or method appropriate to the exercise.
| Level | Description |
|---|---|
| 4 | The submission is highly unique, incorporating novel ideas and displaying inventiveness. Approach, structure, or method is clearly original and justified; rationales for key choices are correct and clear. |
| 3 | The submission is unique, involving some new ideas or improvements, although they are predictable or conventional, or shows a tentative attempt to find new uses for common ideas. Approach or structure is explained with minor gaps in rationale. |
| 2 | Some non-standard choice or structure but without clear rationale, or one plausible original idea stated but not carried through. Does not achieve Level 3. |
| 1 | Purely conventional or template-like response with no stated rationale for approach, structure, or choices. Does not achieve Level 2. |
| 0 | No discernible approach, structure, or submission; or submission does not address how the work was done or what was produced. |
Criterion: Evidence of generating or selecting ideas, methods, or tools beyond the bare minimum (e.g. additional detection features, extra plugins or scripts, richer structure, justified tool or technique choice, or automation that extends depth or reusability).
| Level | Description |
|---|---|
| 4 | Rich extension beyond the minimum: multiple justified choices of methods, tools, structure, or automation; procedure and artifacts align; rationales are correct and complete; serves as a functional solution for the exercise and similar situations. |
| 3 | Clear extension beyond the minimum; rationale for choices present with minor gaps; artifacts support the claimed approach; may be conventional but shows deliberate selection or generation of ideas. |
| 2 | Some use of extra features, tools, or structure but without clear justification or without extending insight; or extension mentioned but not evidenced in artifacts. Does not achieve Level 3. |
| 1 | Only the bare minimum (e.g. minimal required features, no extra tools or structure, no automation or extension). Does not achieve Level 2. |
| 0 | No use of required tools/methods demonstrated; or no procedure or artifacts submitted. |
Criterion: Usefulness of the submission for the exercise-thorough, justified interpretation of output or implications; explanation of why findings or choices matter; meeting exercise needs with technical depth and no extraneous or vague content.
| Level | Description |
|---|---|
| 4 | Thorough, well-justified interpretation or explanation; significance of findings or choices clearly tied to objectives; submission meets exercise needs as a functional solution; technical depth where applicable; no extraneous or vague commentary. |
| 3 | Clear interpretation or explanation with acceptable rationale; technical depth in explaining significance with minor gaps; meets exercise needs but not fully articulated. |
| 2 | Some interpretation or explanation but with little justification; “why it matters” is partial or vague. Does not achieve Level 3. |
| 1 | Only minimal interpretation (e.g. one-line labels or no explanation of significance); no clear link to exercise needs or objectives. Does not achieve Level 2. |
| 0 | No interpretation of output or implications; raw output or unannotated submission only; no commentary on relevance or meaning. |
All universal rubric categories use the same creativity rubric so that no per-exercise creativity rubric is required. Exercises that assess creativity reference the universal rubric and, when needed, use the evidence_location hints below so instructors know where to look.
Evidence_location hints by category: Where to look for the three criteria (originality, extends_beyond_minimum, value) so instructors can apply the universal level descriptions without exercise-specific text. Use the same three criteria for every category; only the “where to look” changes.
| Category | Originality (approach, structure, novelty) | Extends beyond minimum (tools, methods, features) | Value (interpretation, usefulness, why it matters) |
|---|---|---|---|
| yara | Rule structure, choice of detection logic, metadata, use of strings/conditions | Number and type of detection features, use of modules or advanced syntax beyond minimum | Explanation of rule purpose, when it fires, relevance to scenario |
| hunt_query | Query structure, choice of logic (e.g. filters, joins), commented reasoning | Extra columns, aggregations, commented logic, use of supported syntax beyond minimal run | Explanation of what the query finds, why it matters for the hunt |
| jupyter_lab | Notebook structure, choice of analysis steps, order of cells | Cells beyond minimum, visualizations, commented logic, libraries used | Interpretation of results, conclusions, relevance to exercise |
| lab_setup | Order of steps, choice of validation method | Extra validation or documentation beyond minimum | Explanation of what was configured and why it matters |
| fuzzing | Harness design, corpus choice, coverage strategy | Harness features, instrumentation, corpus beyond minimum | Interpretation of crashes/coverage, relevance to target |
| static_analysis | Analysis path, choice of tools or views (e.g. Ghidra) | Exports, annotations, multiple views or scripts | Documented findings, significance of artifacts |
| dynamic_analysis | Runtime analysis strategy, choice of instrumentation | Tracing options, coverage or behavior logging beyond minimum | Documented behavior, interpretation of traces |
| exploit_code | Approach to exploit (e.g. technique, bypass strategy) | Code structure, libraries, mitigations | Explanation of exploit flow, relevance to objective |
| report | Document structure, section choices, argument flow | Sections beyond minimum, methodology detail, sources | Findings, recommendations, link to exercise needs |
| cloud_config | Config design, choice of services or settings | Extra hardening, least-privilege, documentation | Explanation of what was configured and why |
| red_team_operation | Operational approach, TTP selection, planning | Payloads, C2, persistence choices; report structure | action plan/report; link to objective |
| pen_test | Scan/exploit sequence, tool choice | Tools and options used beyond minimum | Documented findings, risk/relevance |
| secure_development | Code structure, hardening approach | Fixes, libraries, patterns beyond minimum | Explanation of fixes and security impact |
| forensics | Analysis path, plugin/step order, hypothesis | Plugins, scripts, tooling beyond imageinfo/pslist | Annotations, interpretation, link to compromise/timeline |
| vulnerability_research | Research approach, triage method | Tools, patch diffing depth, disclosure structure | Documented findings, impact, disclosure |
| capability_development | Tool design, choice of approach | Code features, options, reproducibility | Explanation of what the tool does and when to use it |
| leadership_deliverable | Document structure, metrics/framework choice | Sections, metrics, justification depth | Link to scenario, stakeholder needs, professional tone |
| grc_document | Document structure, scope, roles | Sections, procedures, checklists beyond minimum | Clarity of procedures, link to governance/risk |
| devops_pipeline | Pipeline design, stage choice | Steps, security checks, config beyond minimum | Documentation of steps, build/scan outcome |
| network_forensics | Analysis path, filter/correlation choice | Tools, filters, IOC extraction beyond minimum | Annotations, interpretation, link to incident |
| deobfuscation_artifact | Recovery strategy, technique identification | Recovery method, documentation depth | Explanation of obfuscation and recovered artifact |
| sysadmin_config | Config approach, hardening choices | Settings, GPO/config beyond minimum | Explanation of config and validation |
| hunt_query_and_report | Query + report structure, hunt logic | Query features, report sections beyond minimum | Findings, methodology, recommendations |
Y. R. Kim, J. Yang, Y. Lee and B. Earwood, Assessing Cybersecurity Problem-Solving Skills and Creativity of Engineering Students Through Model-Eliciting Activities Using an Analytic Rubric, in IEEE Access, vol. 12, pp. 5743-5759, 2024, doi: 10.1109/ACCESS.2023.3348554.
{ “id”: “universal_mea_creativity”, “version”: “1.0”, “description”: “Single MEA-style creativity rubric for all categories. Reference by exercise/category to avoid per-exercise creativity definitions.”, “source”: “Assessing Cybersecurity Problem-Solving Skills and Creativity of Engineering Students Through Model-Eliciting Activities Using an Analytic Rubric (IEEE); creativity = novelty + usefulness.”, “scope”: “All categories in category_descriptions.md. Documentation and Presentation covered by critical thinking/execution; creativity = Originality, Generation & Selection of Ideas, Value.”, “usage”: “Set creativity.use = \”universal_mea_creativity\” (or reference this file). Optional: use evidence_location from category_descriptions.md for where to look.”, “overall_score”: “mean of the three criterion scores (0-4).”, “optional_assessment”: true, “general_scale_0_4”: { “4”: { “label”: “Fully accomplishes”, “definition”: “Product/work fully accomplishes the criterion; highly unique (originality) or strong extension/value; clear, justified, complete; rationales correct; no extraneous information.” }, “3”: { “label”: “Accomplishes with minor gaps”, “definition”: “Accomplishes the criterion; unique but predictable/conventional or tentative new uses for common ideas; viable with minor errors or rationale gaps.” }, “2”: { “label”: “Somewhat accomplishes”, “definition”: “Partially meets requirements; gaps, errors, or lacks clarity. Does not achieve Level 3.” }, “1”: { “label”: “Does not meet minimum”, “definition”: “Does not achieve Level 2; critical evidence (results, rationale, required output) missing.” }, “0”: { “label”: “No evidence / No product”, “definition”: “No progress or nothing that resembles a minimal response; no substantive procedure or submission; work does not address the criterion.” } }, “criteria”: { “demonstrates_original_approach”: { “short_name”: “Originality”, “definition”: “Novelty of the product or approach-unexpected or inventive use of ideas, structure, or method appropriate to the exercise.”, “scale_0_4”: { “4”: “The product is highly unique, incorporating novel ideas and displaying inventiveness. Approach, structure, or method is clearly original and justified; rationales for key choices are correct and clear.”, “3”: “The product is unique, involving some new ideas or improvements (predictable or conventional) or a tentative attempt to find new uses for common ideas. Approach or structure is explained with minor gaps in rationale.”, “2”: “Some non-standard choice or structure but without clear rationale, or one plausible original idea stated but not carried through. Does not achieve Level 3.”, “1”: “Purely conventional or template-like response with no stated rationale for approach, structure, or choices. Does not achieve Level 2.”, “0”: “No discernible approach, structure, or product; or submission does not address how the work was done or what was produced.” } }, “extends_analysis_with_custom_or_advanced_tooling”: { “short_name”: “Generation & selection of ideas / Extends beyond minimum”, “definition”: “Evidence of generating or selecting ideas, methods, or tools beyond the bare minimum-e.g. additional features, extra tools or scripts, richer structure, justified tool choice, or automation.”, “scale_0_4”: { “4”: “Rich extension beyond the minimum: multiple justified choices of methods, tools, structure, or automation; procedure and artifacts align; rationales correct and complete; functional solution for exercise and similar situations.”, “3”: “Clear extension beyond the minimum; rationale for choices present with minor gaps; artifacts support the claimed approach; shows deliberate selection or generation of ideas.”, “2”: “Some use of extra features, tools, or structure without clear justification or without extending insight; or extension mentioned but not evidenced in artifacts. Does not achieve Level 3.”, “1”: “Only the bare minimum (minimal required features, no extra tools or structure, no automation or extension). Does not achieve Level 2.”, “0”: “No use of required tools/methods demonstrated; or no procedure or artifacts submitted.” } }, “provides_enhanced_interpretation_through_technical_depth”: { “short_name”: “Value / Usefulness”, “definition”: “Usefulness for the exercise-thorough, justified interpretation of output or implications; explanation of why findings or choices matter; meeting exercise needs with technical depth.”, “scale_0_4”: { “4”: “Thorough, well-justified interpretation or explanation; significance tied to objectives; product meets exercise needs as functional solution; technical depth where applicable; no extraneous or vague commentary.”, “3”: “Clear interpretation or explanation with acceptable rationale; technical depth in explaining significance with minor gaps; meets exercise needs but not fully articulated.”, “2”: “Some interpretation or explanation with little justification; why it matters is partial or vague. Does not achieve Level 3.”, “1”: “Only minimal interpretation or no explanation of significance; no clear link to exercise needs or objectives. Does not achieve Level 2.”, “0”: “No interpretation of output or implications; raw output or unannotated submission only; no commentary on relevance or meaning.” } } } }Below is an interactive rubric viewer demonstrating how the universal rubric applies to a specific exercise. The example uses the Volatility memory analysis task from the Certified Threat Hunter course. It shows the instructor view with submission requirements, critical thinking criteria, creativity criteria, and video submission checks.
Acquire the memory dump from a target machine and analyze it using Volatility.
Objective
Background
Task
Verify these requirements are met before scoring. Unmet requirements may make the submission ineligible for grading.
Confirm the submission meets these requirements.
Grading is for reference only; nothing is saved.